Use Machine Learning To Possibly Become A Millionaire: Predicting The Stock Market? The stock market is one of the most well-known infrastructures through which anyone can potentially make a fortune. If anyone could crack the code to predicting what future stock prices are, they’ll practically rule the world.here’s just one problem. It’s pretty much impossible to accurately predict the future of the stock market.
In this project, we will work with historical data about the stock prices of a publicly listed company. We will implement a mix of machine learning algorithms to predict the future stock price of the company, starting with simple algorithm like linear regression, and then move on to advanced techniques like Auto ARIMA and Neural Networks.
The data set we will be working with is from the New York Stock Exchange (NYSE) and represent the historical prices and other fundamental data points of the S&P 500 from 2010 to the end of 2016. Dataset consists of following files:
prices.csv: raw, as-is daily prices. Most of data spans from 2010 to the end 2016, for companies new on stock market date range is shorter. There have been approx. 140 stock splits in that time, this set doesn’t account for that.
prices-split-adjusted.csv: same as prices, but there have been added adjustments for splits.
securities.csv: general description of each company with division on sectors
fundamentals.csv: metrics extracted from annual SEC 10K fillings (2012-2016), should be enough to derive most of popular fundamental indicators.
The majority of our focus will be on the prices-split-adjusted.csv file, as this contains the adjusted prices for the stocks we will be trying to predict.
Description of fundamentals.csv
| Names | Description |
|---|---|
| Stocks Variable Columns | Categorical - Ticker.Symbol,Period.Ending |
| Stocks Technical Indicator Columns |
Numerical - Accounts.Payable, Accounts.Receivable, Add.l.income.expense.items, After.Tax.ROE, Capital.Expenditures, Capital.Surplus, Cash.Ratio, Cash.and.Cash.Equivalents, Changes.in.Inventories,Common.Stocks, Cost.of.Revenue,Current.Ratio, Deferred.Asset.Charges,Total.Assets, Deferred.Liability.Charges, Depreciation,Earnings.Before.Interest.and.Tax, Earnings.Before.Tax,Net.Income.Adjustments, Effect.of.Exchange.Rate, Equity.Earnings.Loss.Unconsolidated.Subsidiary, Fixed.Assets,Goodwill,Gross.Margin, Gross.Profit,Income.Tax, Intangible.Assets,Interest.Expense, Inventory,Investments,Liabilities, Long.Term.Debt,Long.Term.Investments, Minority.Interest,Misc..Stocks, Net.Borrowings,Net.Cash.Flow , Net.Cash.Flow.Operating,Net.Cash.Flows.Financing, Net.Cash.Flows.Investing,Net.Income, Net.Income.Applicable.to.Common.Shareholders, Net.Income.Cont..,Operations, Net.Receivables,Non.Recurring.Items, Operating.Income,Operating.Margin, Other.Assets,Other.Current.Assets, Other.Current.Liabilities, Other.Equity,Other.Financing.Activities, Other.Investing.Activities,Other.Liabilities, Other.Operating.Activities,Pre.Tax.Margin Other.Operating.Items,Pre.Tax.ROE, Profit.Margin,Quick.Ratio, Research.and.Development, Retained.Earnings,Short.Term.Debt Sale.and.Purchase.of.Stock, Sales..General.and.Admin., Current.Portion.of.Long.Term.Debt, Short.Term.Investments,Equity, Total.Current.Assets,For.Year Total.Current.Liabilities, Total.Equity,Total.Liabilities, Total.Revenue,Treasury.Stock, Earnings.Per.Share, Estimated.Shares.Outstanding |
Description of securities.csv
| Names | Description |
|---|---|
| Ticker.symbol | Categorical - Ticker.Symbol |
| GICS.Sector | Categorical -Industry of the ticker |
| GICS.Sub.Industry | Categorical -Sub.Industry of the ticker |
Description of sprice.csv
| Names | Description |
|---|---|
| Date | Date - Date for which prices are recorded |
| Symbol | Categorical - Ticker.Symbol |
| Open | Numerical - Opening price of the stock on a particular day |
| Close | Numerical - Closing price of the stock on a particular day |
| Low | Numerical -Low price of the stock on a particular day |
| High | Numerical -High price of the stock on a particular day |
| Volume | Numerical -Volume of the stock on a particular day |
Data Structure and manipulation.
| Names | Description |
|---|---|
| Ticker.Symbol | Categorical - Stock Symbol |
| Industry | Categorical - Idustry of the Stock |
| endyear | Date - Year of the revenue |
| quarter | Date - Quarter of the revenue |
| Stock.p | Numerical - Mean of Close price |
| Capital.Expenditures | Numerical - Technical Indicator |
| Cash.Ratio | Numerical - Technical Indicator |
| Cost.of.Revenue | Numerical - Technical Indicator |
| Depreciation | Numerical - Technical Indicator |
| Earnings.Before.Interest.and.Tax | Numerical - Technical Indicator |
| Effect.of.Exchange.Rate | Numerical - Technical Indicator |
| Fixed.Assets | Numerical - Technical Indicator |
| Gross.Profit | Numerical - Technical Indicator |
| Investments | Numerical - Technical Indicator |
| Liabilities | Numerical - Technical Indicator |
| Long.Term.Debt | Numerical - Technical Indicator |
| Long.Term.Investments | Numerical - Technical Indicator |
| Net.Cash.Flow | Numerical - Technical Indicator |
| Net.Income | Numerical - Technical Indicator |
| Pre.Tax.ROE | Numerical - Technical Indicator |
| Profit.Margin | Numerical - Technical Indicator |
| Retained.Earnings | Numerical - Technical Indicator |
| Total.Current.Assets | Numerical - Technical Indicator |
| Total.Current.Liabilities | Numerical - Technical Indicator |
| Total.Equity | Numerical - Technical Indicator |
| Total.Liabilities | Numerical - Technical Indicator |
| Total.Revenue | Numerical - Technical Indicator |
| Earnings.Per.Share | Numerical - Technical Indicator |
| Estimated.Shares.Outstanding | Numerical - Technical Indicator |
Corrplot With Correlation Coefficient
Frequency Table for Industry:
Consumer Discretionary Consumer Staples Energy
120 50 58
Financials Health Care Industrials
10 74 120
Information Technology Materials Real Estate
108 44 14
Utilities
42
Frequency Table for MarketCapIndiactor:
high low medium
292 301 47
Contingency Table for Industry/ MarketCapIndiactor:
high low medium
Consumer Discretionary 46 67 7
Consumer Staples 28 18 4
Energy 32 23 3
Financials 3 4 3
Health Care 45 23 6
Industrials 47 65 8
Information Technology 49 52 7
Materials 17 24 3
Real Estate 6 6 2
Utilities 19 19 4
Proportions Table for Industry/ MarketCapIndiactor:
high low medium
Consumer Discretionary 0.0718750 0.1046875 0.0109375
Consumer Staples 0.0437500 0.0281250 0.0062500
Energy 0.0500000 0.0359375 0.0046875
Financials 0.0046875 0.0062500 0.0046875
Health Care 0.0703125 0.0359375 0.0093750
Industrials 0.0734375 0.1015625 0.0125000
Information Technology 0.0765625 0.0812500 0.0109375
Materials 0.0265625 0.0375000 0.0046875
Real Estate 0.0093750 0.0093750 0.0031250
Utilities 0.0296875 0.0296875 0.0062500
Summary MarketCapIndiactor
Length Class Mode
640 character character
Summary Industry
Consumer Discretionary Consumer Staples Energy
120 50 58
Financials Health Care Industrials
10 74 120
Information Technology Materials Real Estate
108 44 14
Utilities
42
Numerical Summary of Market Cap
Min. 1st Qu. Median Mean 3rd Qu. Max.
2.609e+09 1.030e+10 1.678e+10 3.534e+10 3.393e+10 6.907e+11
Numerical Summary of Market Cap grouped by Industry
# A tibble: 10 x 6
Industry sum mean min max median
<fct> <dbl> <dbl> <dbl> <dbl> <dbl>
1 Consumer Discretion… 3.33e12 27755715066. 4.62e9 2.23e11 1.38e10
2 Consumer Staples 2.74e12 54777333131. 4.69e9 2.53e11 2.83e10
3 Energy 2.44e12 42025301685. 4.57e9 4.16e11 2.27e10
4 Financials 1.55e11 15499103508. 7.49e9 2.69e10 1.60e10
5 Health Care 3.31e12 44710159546. 3.22e9 2.08e11 2.31e10
6 Industrials 3.04e12 25355115660. 4.07e9 1.01e11 1.38e10
7 Information Technol… 5.52e12 51151898108. 2.61e9 6.91e11 1.61e10
8 Materials 9.05e11 20569895953. 4.04e9 5.99e10 1.44e10
9 Real Estate 3.33e11 23773975192. 5.95e9 4.93e10 1.81e10
10 Utilities 8.41e11 20019656246. 3.28e9 5.23e10 1.63e10
[1] "skewness of Industry -0.051643"
[1] "skewness of MarketCapIndiactor 0.476626"
[1] "skewness of Market.Cap 5.468953"
[1] "skewness of Total.Revenue 6.125020"
[1] "skewness of Gross.Profit 5.369591"
Shapiro Wilkes Test
Shapiro-Wilk normality test
data: as.numeric(as.factor(s$Industry))[0:200]
W = 0.90934, p-value = 1.033e-09
###Graphical Summaries
'data.frame': 640 obs. of 29 variables:
$ Ticker.Symbol : Factor w/ 320 levels "AAL","AAPL","ABBV",..: 1 1 2 2 3 3 4 4 5 5 ...
$ Capital.Expenditures : num -5.31e+09 -6.15e+09 -9.57e+09 -1.12e+10 -6.12e+08 ...
$ Cash.Ratio : num 60 51 40 52 74 77 10 10 43 67 ...
$ Cost.of.Revenue : num 1.56e+10 1.11e+10 1.12e+11 1.40e+11 4.43e+09 ...
$ Depreciation : num 1.34e+09 1.49e+09 7.95e+09 1.13e+10 7.86e+08 ...
$ Earnings.Before.Interest.and.Tax: num 4.10e+09 5.50e+09 5.35e+10 7.25e+10 2.76e+09 ...
$ Fixed.Assets : num 2.31e+10 2.75e+10 2.06e+10 2.25e+10 2.48e+09 ...
$ Gross.Profit : num 2.70e+10 2.99e+10 7.05e+10 9.36e+10 1.55e+10 ...
$ Investments : num 1.80e+09 4.43e+08 -9.03e+09 -4.44e+10 3.08e+08 ...
$ Liabilities : num -1.03e+09 -6.33e+08 1.34e+10 1.52e+10 -1.93e+08 ...
$ Long.Term.Debt : num 1.60e+10 1.83e+10 2.90e+10 5.33e+10 1.05e+10 ...
$ Net.Cash.Flow : num -1.46e+08 -6.04e+08 -4.15e+08 7.28e+09 -1.25e+09 ...
$ Pre.Tax.ROE : num 159 82 48 61 136 168 34 44 12 15 ...
$ Profit.Margin : num 7 19 22 23 9 23 0 0 11 22 ...
$ Retained.Earnings : num -8.56e+09 -1.23e+09 8.72e+10 9.23e+10 5.35e+08 ...
$ Total.Current.Assets : num 1.18e+10 9.98e+09 6.85e+10 8.94e+10 1.61e+10 ...
$ Total.Current.Liabilities : num 1.34e+10 1.36e+10 6.34e+10 8.06e+10 1.14e+10 ...
$ Total.Equity : num 2.02e+09 5.64e+09 1.12e+11 1.19e+11 1.74e+09 ...
$ Total.Liabilities : num 4.12e+10 4.28e+10 1.20e+11 1.71e+11 2.58e+10 ...
$ Total.Revenue : num 4.26e+10 4.10e+10 1.83e+11 2.34e+11 2.00e+10 ...
$ Earnings.Per.Share : num 4.02 11.39 6.49 9.28 1.11 ...
$ Estimated.Shares.Outstanding : num 7.17e+08 6.68e+08 6.09e+09 5.75e+09 1.60e+09 ...
$ Industry : Factor w/ 10 levels "Consumer Discretionary",..: 6 6 7 7 5 5 5 5 5 5 ...
$ endyear : Factor w/ 2 levels "2014","2015": 1 2 1 2 1 2 1 2 1 2 ...
$ quarter : Factor w/ 4 levels "1","2","3","4": 4 4 3 3 4 4 3 3 4 4 ...
$ Stock.p : num 38.9 45.1 92.3 120 55.5 ...
$ Year : Factor w/ 2 levels "2014","2015": 1 2 1 2 1 2 1 2 1 2 ...
$ Market.Cap : num 2.79e+10 3.01e+10 5.62e+11 6.91e+11 8.86e+10 ...
$ MarketCapIndiactor : chr "high" "high" "high" "high" ...
R2 value for Market.Cap and Total.Equity
Scatter Plot for Market.Cap & Total.Revenue
R2 value for Market.Cap and Total.Revenue
Statistical Hypothesis Testing can be categorized into two types as below:
Welch Two Sample t-test
data: visualization_data$Market.Cap and visualization_data$Total.Revenue
t = 4.908, df = 1179.8, p-value = 1.049e-06
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
8411277703 19614850898
sample estimates:
mean of x mean of y
35337997368 21324933067
[1] "P value for Market cap & Total revenue 0.000001"
[1] "Group Means for Market cap & Total revenue 35337997367.965248"
[2] "Group Means for Market cap & Total revenue 21324933067.187500"
[1] "Confidence interval for difference for Market cap & Total revenue 8411277703.442698"
[2] "Confidence interval for difference for Market cap & Total revenue 19614850898.112801"
[1] "Confidence level for Market cap & Total revenue 0.950000"
[1] "GOOGL"
We want to do a hypothesis about the daily change of price of stock trading. Price may be volatile for stocks which are trending, but it might also be stable over the long-haul.
Testing to see if daily price change has changed would make sense if we wanted to evaluate stability over two time frames. For example, if we wanted to do a pre-post analysis after a sales / marketing event, then we would test two samples (one before and one after).
Test is two-tailed, meaning we want to know if price increased or decreased.
Once again, we have a very small p-value (near zero), meaning we need to reject the null hypothesis. Mean daily price change is not equal to the baseline price change.Obviously, changing the alpha level makes no difference, because p is so close to zero.
One Sample t-test
data: DataSetGoogle$Price2012
t = -33.718, df = 501, p-value < 2.2e-16
alternative hypothesis: true mean is not equal to 6.241242
95 percent confidence interval:
-0.1912902 0.5170744
sample estimates:
mean of x
0.1628921
From the results of the F test, we can see that the null hypothesis that the variances are equal is rejected, because the p-value is below .05. This means we should use the unequal variance t-test (also known as the Welch.)
The results of this test are also clear. Reject the null hypothesis that that years 2012 adjusted closing prices are less than or equal to years 2014. That is nice to know. We would definitely want to investigate further, but we would surmise that if a company invested five years ago in a long-term position, they would not have greater gains!
F test to compare two variances
data: DataSetGoogle$Price2012 and DataSetGoogle$Price2014
F = 0.32317, num df = 501, denom df = 501, p-value < 2.2e-16
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
0.2711993 0.3851034
sample estimates:
ratio of variances
0.3231715
Two Sample t-test
data: DataSetGoogle$Price2012 and DataSetGoogle$Price2014
t = 1.6032, df = 1002, p-value = 0.1092
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.1310177 1.3005862
sample estimates:
mean of x mean of y
0.1628921 -0.4218921
Welch Two Sample t-test
data: DataSetGoogle$Price2012 and DataSetGoogle$Price2014
t = 1.6032, df = 794.2, p-value = 0.1093
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.1312443 1.3008127
sample estimates:
mean of x mean of y
0.1628921 -0.4218921
[1] "GOOGL"
[1] "AMZN"
Two Sample t-test
data: pairedtestdataset$PriceGoogle and pairedtestdataset$PriceAmazon
t = -0.41406, df = 1002, p-value = 0.6789
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.3699250 0.2410143
sample estimates:
mean of x mean of y
-0.12885399 -0.06439866
Welch Two Sample t-test
data: pairedtestdataset$PriceGoogle and pairedtestdataset$PriceAmazon
t = -0.41406, df = 828.45, p-value = 0.6789
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.3700024 0.2410917
sample estimates:
mean of x mean of y
-0.12885399 -0.06439866
F test to compare two variances
data: pairedtestdataset$PriceGoogle and pairedtestdataset$PriceAmazon
F = 2.688, num df = 501, denom df = 501, p-value < 2.2e-16
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
2.255708 3.203109
sample estimates:
ratio of variances
2.687988
The diagnostic plots of availability with price show residuals in four different ways:
Residuals vs Fitted: This diagnostic plot is an indicator of the Linearity or Non Linearity of the relationship.If there is no perceivable pattern around the central horizontal curve then the relationship is linear.
Normal Q-Q: This diagnostic plot ascertains whether the residuals are normally distributed.If the resiuals are algned to the central diagonal then the residuals follow a straight line.
Scale-Location: This diagnostic is to evaluate the homogeneity of variance of the residuals .If the residuals are spread uniformly around the central line then the residuals are homoscedastic.
Residuals vs Leverage. Used to identify influential cases, that is extreme values that might influence the regression results when included or excluded from the analysis.
[1] "VIF score for all features"
Total.Equity Total.Revenue
3.198600 3.939829
Estimated.Shares.Outstanding Earnings.Before.Interest.and.Tax
2.732357 3.040498
Gross.Profit Stock.p
7.184466 1.062775
+ Fold1.Rep1: intercept=TRUE
- Fold1.Rep1: intercept=TRUE
+ Fold2.Rep1: intercept=TRUE
- Fold2.Rep1: intercept=TRUE
+ Fold3.Rep1: intercept=TRUE
- Fold3.Rep1: intercept=TRUE
+ Fold1.Rep2: intercept=TRUE
- Fold1.Rep2: intercept=TRUE
+ Fold2.Rep2: intercept=TRUE
- Fold2.Rep2: intercept=TRUE
+ Fold3.Rep2: intercept=TRUE
- Fold3.Rep2: intercept=TRUE
+ Fold1.Rep3: intercept=TRUE
- Fold1.Rep3: intercept=TRUE
+ Fold2.Rep3: intercept=TRUE
- Fold2.Rep3: intercept=TRUE
+ Fold3.Rep3: intercept=TRUE
- Fold3.Rep3: intercept=TRUE
Aggregating results
Fitting final model on full training set
Call:
lm(formula = .outcome ~ ., data = dat)
Residuals:
Min 1Q Median 3Q Max
-2.233e+10 -4.448e+09 -9.596e+08 2.379e+09 3.628e+10
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 2.139e+10 2.102e+08 101.771 < 2e-16 ***
Total.Equity 9.649e+08 2.724e+08 3.542 0.000409 ***
Total.Revenue -5.398e+07 3.301e+08 -0.164 0.870121
Estimated.Shares.Outstanding 4.801e+09 3.153e+08 15.226 < 2e-16 ***
Earnings.Before.Interest.and.Tax 3.977e+09 3.150e+08 12.623 < 2e-16 ***
Gross.Profit 3.104e+09 3.696e+08 8.397 < 2e-16 ***
Stock.p 3.654e+09 2.338e+08 15.632 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 8.237e+09 on 1529 degrees of freedom
Multiple R-squared: 0.6548, Adjusted R-squared: 0.6534
F-statistic: 483.3 on 6 and 1529 DF, p-value: < 2.2e-16
RMSE: 4968177423 Error: 0.2309932 R2: 0.9949914
Interpretation of results
R-squared has the useful property that its scale is intuitive: it ranges from zero to one, with zero indicating that the proposed model does not improve prediction over the mean model, and one indicating perfect prediction. Improvement in the regression model results in proportional increases in R-squared.
The RMSE is the square root of the variance of the residuals. It indicates the absolute fit of the model to the data–how close the observed data points are to the model’s predicted values. Whereas R-squared is a relative measure of fit, RMSE is an absolute measure of fit. Lower values of RMSE indicate better fit.
Since error percentage is 23, R2=0.99, RMSE is , it is safe to assume that market cap is affected by “Total.Equity”,“Total.Revenue”, “Estimated.Shares.Outstanding”,“Earnings.Before.Interest.and.Tax”,“Gross.Profit”,“Stock.p”.
Polynomial regression is a special case of linear regression where we fit a polynomial equation on the data with a curvilinear relationship between the target variable and the independent variables.
In a curvilinear relationship, the value of the target variable changes in a non-uniform manner with respect to the predictor (s).
poly function returns or evaluates orthogonal polynomials of degree 1 to degree over the specified set of points x: these are all orthogonal to the constant polynomial of degree 0.
+ Fold1.Rep1: intercept=TRUE
- Fold1.Rep1: intercept=TRUE
+ Fold2.Rep1: intercept=TRUE
- Fold2.Rep1: intercept=TRUE
+ Fold3.Rep1: intercept=TRUE
- Fold3.Rep1: intercept=TRUE
+ Fold1.Rep2: intercept=TRUE
- Fold1.Rep2: intercept=TRUE
+ Fold2.Rep2: intercept=TRUE
- Fold2.Rep2: intercept=TRUE
+ Fold3.Rep2: intercept=TRUE
- Fold3.Rep2: intercept=TRUE
+ Fold1.Rep3: intercept=TRUE
- Fold1.Rep3: intercept=TRUE
+ Fold2.Rep3: intercept=TRUE
- Fold2.Rep3: intercept=TRUE
+ Fold3.Rep3: intercept=TRUE
- Fold3.Rep3: intercept=TRUE
Aggregating results
Fitting final model on full training set
Call:
lm(formula = .outcome ~ ., data = dat)
Residuals:
Min 1Q Median 3Q Max
-1.693e+10 -3.719e+09 -1.128e+09 1.712e+09 4.125e+10
Coefficients:
Estimate Std. Error t value
(Intercept) 2.139e+10 1.893e+08 112.981
`poly(Total.Equity, 2)1` 6.920e+08 2.562e+08 2.701
`poly(Total.Equity, 2)2` -3.021e+08 2.006e+08 -1.506
`poly(Total.Revenue, 2)1` -1.022e+09 3.300e+08 -3.096
`poly(Total.Revenue, 2)2` 4.368e+08 2.210e+08 1.976
`poly(Estimated.Shares.Outstanding, 2)1` 6.839e+09 3.241e+08 21.099
`poly(Estimated.Shares.Outstanding, 2)2` -3.001e+09 2.174e+08 -13.804
`poly(Earnings.Before.Interest.and.Tax, 2)1` 2.823e+09 3.021e+08 9.345
`poly(Earnings.Before.Interest.and.Tax, 2)2` 7.322e+08 2.015e+08 3.633
`poly(Gross.Profit, 2)1` 2.982e+09 3.786e+08 7.876
`poly(Gross.Profit, 2)2` -7.289e+08 2.282e+08 -3.195
`poly(Stock.p, 2)1` 4.900e+09 2.256e+08 21.724
`poly(Stock.p, 2)2` -2.355e+09 2.020e+08 -11.659
Pr(>|t|)
(Intercept) < 2e-16 ***
`poly(Total.Equity, 2)1` 0.00700 **
`poly(Total.Equity, 2)2` 0.13233
`poly(Total.Revenue, 2)1` 0.00199 **
`poly(Total.Revenue, 2)2` 0.04833 *
`poly(Estimated.Shares.Outstanding, 2)1` < 2e-16 ***
`poly(Estimated.Shares.Outstanding, 2)2` < 2e-16 ***
`poly(Earnings.Before.Interest.and.Tax, 2)1` < 2e-16 ***
`poly(Earnings.Before.Interest.and.Tax, 2)2` 0.00029 ***
`poly(Gross.Profit, 2)1` 6.37e-15 ***
`poly(Gross.Profit, 2)2` 0.00143 **
`poly(Stock.p, 2)1` < 2e-16 ***
`poly(Stock.p, 2)2` < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 7.42e+09 on 1523 degrees of freedom
Multiple R-squared: 0.721, Adjusted R-squared: 0.7188
F-statistic: 327.9 on 12 and 1523 DF, p-value: < 2.2e-16
RMSE: 5761789130 Error: 0.2309932 R2: 0.9938427
'data.frame': 640 obs. of 23 variables:
$ Capital.Expenditures : num -5.31e+09 -6.15e+09 -9.57e+09 -1.12e+10 -6.12e+08 ...
$ Cash.Ratio : num 60 51 40 52 74 77 10 10 43 67 ...
$ Cost.of.Revenue : num 1.56e+10 1.11e+10 1.40e+10 1.40e+10 4.43e+09 ...
$ Depreciation : num 1.34e+09 1.49e+09 1.14e+09 1.14e+09 7.86e+08 ...
$ Earnings.Before.Interest.and.Tax: num 4.10e+09 2.43e+09 2.43e+09 2.43e+09 2.76e+09 ...
$ Fixed.Assets : num 2.31e+10 9.51e+09 2.06e+10 2.25e+10 2.48e+09 ...
$ Gross.Profit : num 7.33e+09 7.33e+09 7.33e+09 7.33e+09 7.33e+09 ...
$ Investments : num 1.80e+09 4.43e+08 -9.03e+09 -4.44e+10 3.08e+08 ...
$ Liabilities : num 1.41e+08 1.41e+08 1.41e+08 1.41e+08 -1.93e+08 ...
$ Long.Term.Debt : num 1.60e+10 6.61e+09 6.61e+09 6.61e+09 1.05e+10 ...
$ Net.Cash.Flow : num -1.46e+08 -6.04e+08 -4.15e+08 7.28e+09 -1.25e+09 ...
$ Pre.Tax.ROE : num 159 82 48 61 136 168 34 44 12 15 ...
$ Profit.Margin : num 7 19 22 23 9 23 0 0 11 22 ...
$ Retained.Earnings : num -8.56e+09 -1.23e+09 9.73e+09 9.73e+09 5.35e+08 ...
$ Total.Current.Assets : num 1.18e+10 9.98e+09 6.85e+10 8.94e+10 1.61e+10 ...
$ Total.Current.Liabilities : num 1.34e+10 1.36e+10 6.34e+10 8.06e+10 1.14e+10 ...
$ Total.Equity : num 2.02e+09 5.64e+09 9.41e+09 9.41e+09 1.74e+09 ...
$ Total.Liabilities : num 4.12e+10 4.28e+10 1.69e+10 1.69e+10 2.58e+10 ...
$ Total.Revenue : num 2.13e+10 2.13e+10 2.13e+10 2.13e+10 2.00e+10 ...
$ Earnings.Per.Share : num 4.02 3.27 6.49 3.27 1.11 ...
$ Estimated.Shares.Outstanding : num 7.17e+08 6.68e+08 5.64e+08 5.64e+08 5.64e+08 ...
$ Stock.p : num 38.9 45.1 92.3 120 55.5 ...
$ Market.Cap : num 2.79e+10 3.01e+10 3.53e+10 3.53e+10 3.53e+10 ...
Call:
lm(formula = Market.Cap ~ bs(Total.Equity, knots = knots) + bs(Total.Revenue,
knots = knots) + bs(Estimated.Shares.Outstanding, knots = knots) +
bs(Earnings.Before.Interest.and.Tax, knots = knots) + bs(Gross.Profit,
knots = knots), data = splineDatasetSubTrain)
Residuals:
Min 1Q Median 3Q Max
-2.800e+10 -4.215e+09 -6.977e+08 2.852e+09 3.430e+10
Coefficients: (12 not defined because of singularities)
Estimate Std. Error
(Intercept) 3.427e+10 6.318e+09
bs(Total.Equity, knots = knots)1 -1.727e+10 6.381e+09
bs(Total.Equity, knots = knots)2 -5.320e+09 3.911e+09
bs(Total.Equity, knots = knots)3 -8.529e+09 5.238e+09
bs(Total.Equity, knots = knots)4 -8.703e+09 4.685e+09
bs(Total.Equity, knots = knots)5 -9.102e+09 5.085e+09
bs(Total.Equity, knots = knots)6 NA NA
bs(Total.Equity, knots = knots)7 NA NA
bs(Total.Revenue, knots = knots)1 -2.704e+09 2.740e+09
bs(Total.Revenue, knots = knots)2 -7.207e+09 1.720e+09
bs(Total.Revenue, knots = knots)3 -5.667e+09 2.841e+09
bs(Total.Revenue, knots = knots)4 -2.484e+09 3.860e+09
bs(Total.Revenue, knots = knots)5 -5.955e+09 5.374e+09
bs(Total.Revenue, knots = knots)6 -1.659e+10 4.956e+09
bs(Total.Revenue, knots = knots)7 NA NA
bs(Estimated.Shares.Outstanding, knots = knots)1 1.295e+09 2.767e+09
bs(Estimated.Shares.Outstanding, knots = knots)2 1.918e+10 2.905e+09
bs(Estimated.Shares.Outstanding, knots = knots)3 9.242e+08 2.432e+09
bs(Estimated.Shares.Outstanding, knots = knots)4 -5.107e+09 5.035e+09
bs(Estimated.Shares.Outstanding, knots = knots)5 NA NA
bs(Estimated.Shares.Outstanding, knots = knots)6 NA NA
bs(Estimated.Shares.Outstanding, knots = knots)7 NA NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)1 -4.189e+10 8.942e+09
bs(Earnings.Before.Interest.and.Tax, knots = knots)2 1.181e+10 3.244e+09
bs(Earnings.Before.Interest.and.Tax, knots = knots)3 -3.521e+08 5.359e+09
bs(Earnings.Before.Interest.and.Tax, knots = knots)4 3.307e+09 1.027e+10
bs(Earnings.Before.Interest.and.Tax, knots = knots)5 NA NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)6 NA NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)7 NA NA
bs(Gross.Profit, knots = knots)1 1.185e+10 2.733e+09
bs(Gross.Profit, knots = knots)2 1.652e+10 2.554e+09
bs(Gross.Profit, knots = knots)3 1.585e+10 2.959e+09
bs(Gross.Profit, knots = knots)4 1.685e+10 2.868e+09
bs(Gross.Profit, knots = knots)5 NA NA
bs(Gross.Profit, knots = knots)6 NA NA
bs(Gross.Profit, knots = knots)7 NA NA
t value Pr(>|t|)
(Intercept) 5.424 6.77e-08 ***
bs(Total.Equity, knots = knots)1 -2.706 0.006888 **
bs(Total.Equity, knots = knots)2 -1.360 0.173932
bs(Total.Equity, knots = knots)3 -1.628 0.103653
bs(Total.Equity, knots = knots)4 -1.858 0.063417 .
bs(Total.Equity, knots = knots)5 -1.790 0.073660 .
bs(Total.Equity, knots = knots)6 NA NA
bs(Total.Equity, knots = knots)7 NA NA
bs(Total.Revenue, knots = knots)1 -0.987 0.323903
bs(Total.Revenue, knots = knots)2 -4.190 2.95e-05 ***
bs(Total.Revenue, knots = knots)3 -1.995 0.046232 *
bs(Total.Revenue, knots = knots)4 -0.643 0.520074
bs(Total.Revenue, knots = knots)5 -1.108 0.267934
bs(Total.Revenue, knots = knots)6 -3.346 0.000839 ***
bs(Total.Revenue, knots = knots)7 NA NA
bs(Estimated.Shares.Outstanding, knots = knots)1 0.468 0.639956
bs(Estimated.Shares.Outstanding, knots = knots)2 6.602 5.59e-11 ***
bs(Estimated.Shares.Outstanding, knots = knots)3 0.380 0.703998
bs(Estimated.Shares.Outstanding, knots = knots)4 -1.014 0.310574
bs(Estimated.Shares.Outstanding, knots = knots)5 NA NA
bs(Estimated.Shares.Outstanding, knots = knots)6 NA NA
bs(Estimated.Shares.Outstanding, knots = knots)7 NA NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)1 -4.684 3.06e-06 ***
bs(Earnings.Before.Interest.and.Tax, knots = knots)2 3.640 0.000282 ***
bs(Earnings.Before.Interest.and.Tax, knots = knots)3 -0.066 0.947627
bs(Earnings.Before.Interest.and.Tax, knots = knots)4 0.322 0.747503
bs(Earnings.Before.Interest.and.Tax, knots = knots)5 NA NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)6 NA NA
bs(Earnings.Before.Interest.and.Tax, knots = knots)7 NA NA
bs(Gross.Profit, knots = knots)1 4.336 1.54e-05 ***
bs(Gross.Profit, knots = knots)2 6.471 1.31e-10 ***
bs(Gross.Profit, knots = knots)3 5.355 9.87e-08 ***
bs(Gross.Profit, knots = knots)4 5.875 5.18e-09 ***
bs(Gross.Profit, knots = knots)5 NA NA
bs(Gross.Profit, knots = knots)6 NA NA
bs(Gross.Profit, knots = knots)7 NA NA
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 8.429e+09 on 1512 degrees of freedom
Multiple R-squared: 0.6425, Adjusted R-squared: 0.637
F-statistic: 118.1 on 23 and 1512 DF, p-value: < 2.2e-16
RMSE: 4626494209 Error: 0.2151068 R2: 0.8751914
Why Use GAM?Relationships between the individual predictors and the dependent variable follow smooth patterns that can be linear or nonlinear. + Mathematically speaking, GAM is an additive modeling technique where the impact of the predictive variables is captured through smooth functions which—depending on the underlying patterns in the data—can be nonlinear. + GAM can capture common nonlinear patterns that a classic linear model would miss. + GAM framework allows us to control smoothness of the predictor functions to prevent overfitting. By controlling the wiggliness of the predictor functions, we can directly tackle the bias/variance tradeoff.
Family: gaussian
Link function: identity
Formula:
Market.Cap ~ Total.Equity + Earnings.Before.Interest.and.Tax
Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 2.173e-09 5.850e-11 37.14 <2e-16 ***
Total.Equity 1.052e+00 5.873e-02 17.92 <2e-16 ***
Earnings.Before.Interest.and.Tax 8.172e+00 2.327e-01 35.11 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Rank: 2/3
R-sq.(adj) = 0.461 Deviance explained = 43.4%
GCV = 1.1162e+20 Scale est. = 1.1147e+20 n = 1536
RMSE: 6937151743 Error: 0.354476 R2: 0.6515361
Method: GCV Optimizer: magic
Model required no smoothing parameter selectionModel rank = 2 / 3
Family: gaussian
Link function: identity
Formula:
Market.Cap ~ s(Total.Equity, bs = "cr") + s(Total.Revenue, bs = "cr") +
s(Earnings.Before.Interest.and.Tax, bs = "cr")
Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 2.144e+10 3.458e+08 62 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Approximate significance of smooth terms:
edf Ref.df F p-value
s(Total.Equity) 8.091 8.625 6.858 < 2e-16 ***
s(Total.Revenue) 4.292 5.027 4.697 0.000331 ***
s(Earnings.Before.Interest.and.Tax) 8.418 8.829 22.214 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
R-sq.(adj) = 0.607 Deviance explained = 62%
GCV = 7.9235e+19 Scale est. = 7.6536e+19 n = 640
Method: GCV Optimizer: magic
Smoothing parameter selection converged after 7 iterations.
The RMS GCV score gradient at convergence was 2.367604e+13 .
The Hessian was positive definite.
Model rank = 28 / 28
Basis dimension (k) checking results. Low p-value (k-index<1) may
indicate that k is too low, especially if edf is close to k'.
k' edf k-index p-value
s(Total.Equity) 9.00 8.09 1.07 0.956
s(Total.Revenue) 9.00 4.29 0.95 0.088 .
s(Earnings.Before.Interest.and.Tax) 9.00 8.42 1.03 0.798
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
RMSE: 5328801306 Error: 0.2722922 R2: 0.9062193
XGBoost is an implementation of the Gradient Boosted Decision Trees algorithm.
XGBoost supports various objective functions, including regression, classification and ranking.
XGBoost gives among the best performances in many machine learning applications. It is optimized gradient-boosting machine learning library. The core algorithm is parallelizable and hence it can use all the processing power of your machine and the machines in your cluster. In R, according to the package documentation, since the package can automatically do parallel computation on a single machine, it could be more than 10 times faster than existing gradient boosting packages.
XGBoost shines when we have lots of training data where the features are numeric or mixture of numeric and categorical fields. It is also important to note that xgboost is not the best algorithm out there when all the features are categorical or when the number of rows is less than the number of fields (columns).
The data argument in the xgboost R function is for the input features dataset. It accepts a matrix, dgCMatrix, or local data file. The nrounds argument refers to the max number of iterations (i.e. the number of trees added to the model).
There are different hyperparameters that we can tune and the parametres are different from baselearner to baselearner. In tree based learners, which are the most common ones in xgboost applications, the following are the most commonly tuned hyperparameters:
learning rate: learning rate/eta- governs how quickly the model fits the residual error using additional base learners. If it is a smaller learning rate, it will need more boosting rounds, hence more time, to achieve the same reduction in residual error as one with larger learning rate. Typically, it lies between 0.01 - 0.3
The three hyperparameters below are regularization hyperparameters.
gamma: min loss reduction to create new tree split. default = 0 means no regularization.
lambda: L2 reg on leaf weights. Equivalent to Ridge regression.
alpha: L1 reg on leaf weights. Equivalent to Lasso regression.
max_depth: max depth per tree. This controls how deep our tree can grow. The Larger the depth, more complex the model will be and higher chances of overfitting. Larger data sets require deep trees to learn the rules from data. Default = 6.
subsample: % samples used per tree. This is the fraction of the total training set that can be used in any boosting round. Low value may lead to underfitting issues. A very high value can cause over-fitting problems.
colsample_bytree: % features used per tree. This is the fraction of the number of columns that we can use in any boosting round. A smaller value is an additional regularization and a larger value may be cause overfitting issues.
n_estimators: number of estimators (base learners). This is the number of boosting rounds.
[17:36:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:36:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:37:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
MSE: 3.852715e+20 MAE: 18121969088 RMSE: 19628334514
[1] 3988987596
[1] 0.2038301
[1] 0.8701342
RMSE: 3988987596 Error: 0.2038301 R2: 0.8701342
| X.Algorithm. | X.RMSE. | X.R2. | X.Error. |
|---|---|---|---|
| Algorithm | RMSE | R2 | Error |
| ExtremeGradinetBoosting | 3988987596.3155 | 0.870134218809127 | 0.203830128687742 |
| Cubic Regression Spline | 5328801306.20932 | 0.906219318025209 | 0.272292211938517 |
| GAM | 6937151742.56324 | 0.651536114257158 | 0.354476041419414 |
| Spline Regression | 4626494208.87458 | 0.875191434305949 | 0.215106804884498 |
| PolynomialRegression | 0.99384270252873 | 0.230993214992809 | 5761789129.92617 |
As such, we provide evidence suggesting that stock technical indicator has influence in stock market market cap. PolynomialRegression outperforms other model with lesser error and with R2=0.994 the model completely fit.
Why Use GAM?Relationships between the individual predictors and the dependent variable follow smooth patterns that can be linear or nonlinear. + Mathematically speaking, GAM is an additive modeling technique where the impact of the predictive variables is captured through smooth functions which—depending on the underlying patterns in the data—can be nonlinear. + GAM can capture common nonlinear patterns that a classic linear model would miss. + GAM framework allows us to control smoothness of the predictor functions to prevent overfitting. By controlling the wiggliness of the predictor functions, we can directly tackle the bias/variance tradeoff.
Family: gaussian
Link function: identity
Formula:
Market.Cap ~ Total.Equity + Earnings.Before.Interest.and.Tax
Parametric coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 2.173e-09 5.850e-11 37.14 <2e-16 ***
Total.Equity 1.052e+00 5.873e-02 17.92 <2e-16 ***
Earnings.Before.Interest.and.Tax 8.172e+00 2.327e-01 35.11 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Rank: 2/3
R-sq.(adj) = 0.461 Deviance explained = 43.4%
GCV = 1.1162e+20 Scale est. = 1.1147e+20 n = 1536
RMSE: 6937151743 Error: 0.354476 R2: 0.6515361
Method: GCV Optimizer: magic
Model required no smoothing parameter selectionModel rank = 2 / 3
XGBoost is an implementation of the Gradient Boosted Decision Trees algorithm.
XGBoost supports various objective functions, including regression, classification and ranking.
XGBoost gives among the best performances in many machine learning applications. It is optimized gradient-boosting machine learning library. The core algorithm is parallelizable and hence it can use all the processing power of your machine and the machines in your cluster. In R, according to the package documentation, since the package can automatically do parallel computation on a single machine, it could be more than 10 times faster than existing gradient boosting packages.
XGBoost shines when we have lots of training data where the features are numeric or mixture of numeric and categorical fields. It is also important to note that xgboost is not the best algorithm out there when all the features are categorical or when the number of rows is less than the number of fields (columns).
The data argument in the xgboost R function is for the input features dataset. It accepts a matrix, dgCMatrix, or local data file. The nrounds argument refers to the max number of iterations (i.e. the number of trees added to the model).
There are different hyperparameters that we can tune and the parametres are different from baselearner to baselearner. In tree based learners, which are the most common ones in xgboost applications, the following are the most commonly tuned hyperparameters:
learning rate: learning rate/eta- governs how quickly the model fits the residual error using additional base learners. If it is a smaller learning rate, it will need more boosting rounds, hence more time, to achieve the same reduction in residual error as one with larger learning rate. Typically, it lies between 0.01 - 0.3
The three hyperparameters below are regularization hyperparameters.
gamma: min loss reduction to create new tree split. default = 0 means no regularization.
lambda: L2 reg on leaf weights. Equivalent to Ridge regression.
alpha: L1 reg on leaf weights. Equivalent to Lasso regression.
max_depth: max depth per tree. This controls how deep our tree can grow. The Larger the depth, more complex the model will be and higher chances of overfitting. Larger data sets require deep trees to learn the rules from data. Default = 6.
subsample: % samples used per tree. This is the fraction of the total training set that can be used in any boosting round. Low value may lead to underfitting issues. A very high value can cause over-fitting problems.
colsample_bytree: % features used per tree. This is the fraction of the number of columns that we can use in any boosting round. A smaller value is an additional regularization and a larger value may be cause overfitting issues.
n_estimators: number of estimators (base learners). This is the number of boosting rounds.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:38:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:39:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
[17:40:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
MSE: 3.852715e+20 MAE: 18121969088 RMSE: 19628334514
[1] 3988987596
[1] 0.2038301
[1] 0.8701342
RMSE: 3988987596 Error: 0.2038301 R2: 0.8701342
[1] "BAC"
[1] "RSI3" "EMAcross" "MACDsignal" "Stochastic" "Class"
Data slicing is a step to split data into train and test set. Training data set can be used specifically for our model building. Test dataset should not be mixed up while building model. Even during standardization, we should not standardize our test set.75 percent contributes to training & 25 percent of data contributes to test.
Training the Decision Tree classifier with criterion as information gain
We are setting 3 parameters of trainControl() method. The “method” parameter holds the details about resampling method. We can set “method” to use repeatedcv i.e, repeated cross-validation.
The “number” parameter holds the number of resampling iterations. The “repeats” parameter contains the complete sets of folds to compute for our repeated cross-validation. We are using setting number =10 and repeats =3. This trainControl() methods returns a list. We are going to pass this on our train() method.
To select the specific strategy for decision tree, we need to pass a parameter “parms” in our train() method. It should contain a list of parameters for our rpart method. For splitting criterions, we need to add a “split” parameter with values either “information” for information gain & “gini” for gini index. We are using information gain as a criterion.
CART
3908 samples
4 predictor
2 classes: 'DOWN', 'UP'
Pre-processing: centered (4), scaled (4)
Resampling: Cross-Validated (10 fold, repeated 3 times)
Summary of sample sizes: 3517, 3518, 3517, 3517, 3517, 3517, ...
Resampling results across tuning parameters:
cp Accuracy Kappa
0.001388889 0.5141566 0.0301167276
0.001562500 0.5145822 0.0308771188
0.001822917 0.5152644 0.0322216484
0.001909722 0.5154349 0.0325509692
0.002083333 0.5159460 0.0335094058
0.002343750 0.5170542 0.0345975203
0.002951389 0.5177349 0.0358283997
0.003645833 0.5188436 0.0377622413
0.018229167 0.5058795 0.0006184276
0.019531250 0.5053680 -0.0004969420
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was cp = 0.003645833.
#### Decision Tree Prediction
Now, our model is trained with cp = 0.003645833.Accuracy of the model is 0.5141566.
Confusion Matrix and Statistics
Reference
Prediction DOWN UP
DOWN 317 279
UP 345 360
Accuracy : 0.5204
95% CI : (0.4928, 0.5478)
No Information Rate : 0.5088
P-Value [Acc > NIR] : 0.210682
Kappa : 0.0422
Mcnemar's Test P-Value : 0.009266
Sensitivity : 0.4789
Specificity : 0.5634
Pos Pred Value : 0.5319
Neg Pred Value : 0.5106
Prevalence : 0.5088
Detection Rate : 0.2437
Detection Prevalence : 0.4581
Balanced Accuracy : 0.5211
'Positive' Class : DOWN
kNN algorithm is a non-parametric algorithm that can be used for either classification or regression. Non-parametric means that it makes no assumption about the underlying data or its distribution.
For each data point, the algorithm finds the k closest observations, and then classifies the data point to the majority. Usually, the k closest observations are defined as the ones with the smallest Euclidean distance to the data point under consideration.
K-Nearest Neighbors computes the likelihood of each share closing price based on other technical indicator.
Data Model which we will use for ML is listed below. Here, the 3 columns represent the closing price,opening price,High price, volume of the stock on the given date.
The Increase column represents whether the price of stock rose or fell as compared to the previous day.
We are using caret’s KNN to sample the data (for training and testing), preprocessing, evaluating the model etc.,
Step 1: Data Splicing
Step 2:Data Pre-Processing With Caret
Step 3:Model Training and Tuning
colnames(bny_raw)
1 Date
2 Open
3 increase
4 Volume
5 Close
6 High
Step 4:How to choose value for K to improve performance
k-Nearest Neighbors
8993 samples
5 predictor
2 classes: 'FALSE', 'TRUE'
Pre-processing: centered (5), scaled (5)
Resampling: Cross-Validated (10 fold, repeated 3 times)
Summary of sample sizes: 8093, 8094, 8093, 8094, 8094, 8095, ...
Resampling results across tuning parameters:
k Accuracy Kappa
5 0.5257777 0.02578255
7 0.5287424 0.02899955
9 0.5311870 0.03274320
11 0.5307430 0.02950820
13 0.5333738 0.03257598
15 0.5386393 0.04107065
17 0.5403822 0.04288474
19 0.5420874 0.04482209
21 0.5460535 0.05178726
23 0.5522058 0.06325600
25 0.5534678 0.06464222
27 0.5560607 0.06852801
29 0.5552093 0.06542168
31 0.5540608 0.06233631
33 0.5585827 0.07069766
35 0.5597692 0.07214944
37 0.5595463 0.07104593
39 0.5581399 0.06758583
41 0.5570294 0.06480353
43 0.5573246 0.06410129
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was k = 35.
Confusion Matrix and Statistics
Reference
Prediction FALSE TRUE
FALSE 426 446
TRUE 895 1229
Accuracy : 0.5524
95% CI : (0.5344, 0.5703)
No Information Rate : 0.5591
P-Value [Acc > NIR] : 0.7748
Kappa : 0.0583
Mcnemar's Test P-Value : <2e-16
Sensitivity : 0.3225
Specificity : 0.7337
Pos Pred Value : 0.4885
Neg Pred Value : 0.5786
Prevalence : 0.4409
Detection Rate : 0.1422
Detection Prevalence : 0.2911
Balanced Accuracy : 0.5281
'Positive' Class : FALSE
Accuracy: 0.5524032
K-means clustering is an unsupervised learning method used to uncover non-random structures in your data.
It does so by creating labels for your training data, where each label is a cluster.
An ideal cluster analysis groups points together on a scatterplot, with maximized space (seperation) between groups, and minimized space (cohesion) between points in the same group.
Groups can be centered around a non-real point (k-means).
Challenge: Group stock based on performances
Perform k-means clustering of the New York Stock Exchange (NYSE) dataset that has more than 9,211,031 NYSE trade data. Columns from 1 to 7 are:
Choose K Value
Use columns 2 from the input data and perform the k-means clustering with k = 0 to 15. Set the maximum number of iterations to 10,000.
Use Within Cluster Sum of Squares as a measure to choose best K , which measures the squared average distance of all the points within a cluster to the cluster centroid. To calculate WCSS, you first find the Euclidean distance (see figure below) between a given point and the centroid to which it is assigned. You then iterate this process for all points in the cluster, and then sum the values for the cluster and divide by the number of points. Finally, you calculate the average across all clusters. This will give you the average WCSS.
From the below plot we can see K=7 will not overfit/underfit the model.
Modelling
Length Class Mode
cluster 33333 -none- numeric
centers 14 -none- numeric
totss 1 -none- numeric
withinss 7 -none- numeric
tot.withinss 1 -none- numeric
betweenss 1 -none- numeric
size 7 -none- numeric
iter 1 -none- numeric
ifault 1 -none- numeric
Centers: 26.77171 17.36502 298.0009 84.80548 14.4589 46.8826 14.77272 8.621914 -79.9289 275.3313 64.27364 -31.47357 27.74873 -8.16676
No Of Records in Cluster: 10464 681 23 1032 5088 5087 10958
KNN is a lazy learner where generalization of the training data is delayed until a query is made to the system. Which means knn starts working only when you trigger it to, thus lazy learning methods can construct a different approximation or result to the target function for each encountered query.
It is a good method for online learning but it requires a possibly large amount of memory to store the data, and each request involves starting the identification of a local model from scratch.
The goal of this project is to predict the future stock price of Expedia using various predictive forecasting models and then analysing the models using ARIMA.
The dataset for Expedia stocks is obtained from Yahoo Finance using Quantmod package in R.
The timeline of the data is from 2019 till present day(11/26/2020).
We shall also try and understand the impact of COVID-19 disaster on the stock prices of Expedia.
Forecasting
Data Preparation
Importing the data : We obtain the data of Expedia from “2019-07-01” to “2020-11-26” of Expedia Stock price for our analysis using the quantmod package. To analyse the impact of COVID-19 on the Expedia Stock price, we take two sets of data from the quantmod package.
Data from “2019-07-01” - “2020-03-28” is data before covid.
Data from “2020-04-01” - “till date” is data before covid.
All the analysis and the models will be made on both the datasets to analyse the impact of COVID-19, if any.
[1] "EXPE"
[1] "EXPE"
colnames(google_data_after_covid)
1 EXPE.Open
2 EXPE.High
3 EXPE.Low
4 EXPE.Close
5 EXPE.Volume
6 EXPE.Adjusted
Series: tsData_before_covid_close
ARIMA(3,1,2)
Box Cox transformation: lambda= -0.5522943
Coefficients:
ar1 ar2 ar3 ma1 ma2
0.9436 -0.2552 -0.2793 -0.9659 0.7154
s.e. 0.1333 0.1520 0.0991 0.1174 0.1094
sigma^2 estimated as 1.264e-05: log likelihood=791.34
AIC=-1570.69 AICc=-1570.22 BIC=-1551.3
Training set error measures:
ME RMSE MAE MPE MAPE MASE
Training set -0.3468986 3.779995 2.064975 -0.4697368 2.119611 1.117624
ACF1
Training set -0.01611792
Series: tsData_after_covid_close
ARIMA(1,1,0) with drift
Box Cox transformation: lambda= 1.049021
Coefficients:
ar1 drift
-0.0993 0.5238
s.e. 0.0785 0.3099
sigma^2 estimated as 19.12: log likelihood=-470.79
AIC=947.57 AICc=947.73 BIC=956.86
Training set error measures:
ME RMSE MAE MPE MAPE MASE
Training set -0.002349432 3.471361 2.361182 -0.07665327 2.827689 0.9868023
ACF1
Training set 0.002088858
From the auto.arima function, we conclude the following models for the two datasets:
After obtaining the model, we then perform residual diagnostics for each of the fitted models.
From the residual plot , we can confirm that the residual has a mean of 0 and the variance is constant as well . The ACF is 0 for lag> 0 , and the PACF is 0 as well.
So, we can say that the residual behaves like white noise and conclude that the models ARIMA(3,1,2) and ARIMA(1,1,0) fits the data well. Alternatively, we can also test at a significance level using the Box-Ljung Test.
Try to find out the pattern in the residuals of the chosen model by plotting the ACF of the residuals, and doing a portmanteau test. We need to try modified models if the plot doesn’t look like white noise.
Once the residuals look like white noise, calculate forecasts.
Box-Ljung test is a test of independence at all lags up to the one specified. Instead of testing randomness at each distinct lag, it tests the “overall” randomness based on a number of lags, and is therefore a portmanteau test. It is applied to the residuals of a fitted ARIMA model, not the original series, and in such applications the hypothesis actually being tested is that the residuals from the ARIMA model have no autocorrelation.
The ACF of the residuals shows no significant autocorrelations.
The p-values for the Ljung-Box Q test all are well above 0.05, indicating “non-significance.” The values are normal as they rest on a line and aren’t all over the place.
Augmented Dickey-Fuller Test
data: tsData_before_covid_close
Dickey-Fuller = -2.1153, Lag order = 5, p-value = 0.5279
alternative hypothesis: stationary
Augmented Dickey-Fuller Test
data: tsData_after_covid_close
Dickey-Fuller = -2.4665, Lag order = 5, p-value = 0.3817
alternative hypothesis: stationary
From the above ADF tests, we can conclude the following:
For the dataset before COVID-19, the ADF tests gives a p-value of 0.5279which is greater than 0.05, thus implying that the time series data is not stationary.
For the dataset after COVID-19, the ADF tests gives a p-value of 0.3817 which is lesser greater 0.05, thus implying that the time series data is not stationary.
KPSS Test for Level Stationarity
data: tsData_before_covid_close
KPSS Level = 2.3907, Truncation lag parameter = 4, p-value = 0.01
KPSS Test for Level Stationarity
data: tsData_after_covid_close
KPSS Level = 2.6532, Truncation lag parameter = 4, p-value = 0.01
From the above KPSS tests, we can conclude the following:
For the dataset before COVID-19, the KPSS tests gives a p-value of 0.01 which is less than 0.05, thus implying that the time series data is not stationary.
For the dataset after COVID-19, the KPSS tests gives a p-value of 0.01 which is less than 0.05, thus implying that the time series data is not stationary.
Thus, we can conclude from the above tests that the time series data is not stationary.
Forecasting with ARIMA Models
Forecast errors for before covid dataset
Holt Winters
To make forecasts using simple exponential smoothing in R, we can fit a simple exponential smoothing predictive model using the “HoltWinters()” function in R. To use HoltWinters() for simple exponential smoothing, we need to set the parameters beta=FALSE and gamma=FALSE in the HoltWinters() function (the beta and gamma parameters are used for Holt’s exponential smoothing, or Holt-Winters exponential smoothing, as described below).
HoltWinters() function returns a list variable, that contains several named elements.The output of HoltWinters() tells us that the estimated value of the alpha parameter is about 0.982813.
We can plot the original time series.
Holt-Winters exponential smoothing with trend and without seasonal component.
Call:
HoltWinters(x = tsData_before_covid_close, gamma = FALSE)
Smoothing parameters:
alpha: 0.982813
beta : 0.02381423
gamma: FALSE
Coefficients:
[,1]
a 60.091954
b -0.869578
[1] "skirtsseriesforecasts$SSE"
[1] 2798.291
Point Forecast Lo 80 Hi 80 Lo 95 Hi 95
189 59.22238 54.261714 64.18304 51.6356981 66.80905
190 58.35280 51.315518 65.39008 47.5902068 69.11539
191 57.48322 48.787761 66.17868 44.1846635 70.78178
192 56.61364 46.469767 66.75752 41.0999238 72.12736
193 55.74406 44.280562 67.20757 38.2121512 73.27598
194 54.87449 42.178858 67.57011 35.4581988 74.29077
195 54.00491 40.140333 67.86948 32.8008698 75.20895
196 53.13533 38.149294 68.12137 30.2161653 76.05449
197 52.26575 36.194958 68.33655 27.6875932 76.84391
198 51.39617 34.269562 68.52279 25.2032811 77.58907
199 50.52660 32.367315 68.68588 22.7543724 78.29882
200 49.65702 30.483773 68.83026 20.3340704 78.97997
201 48.78744 28.615447 68.95943 17.9370389 79.63784
202 47.91786 26.759545 69.07618 15.5590078 80.27672
203 47.04828 24.913796 69.18277 13.1965057 80.90006
204 46.17871 23.076330 69.28108 10.8466714 81.51074
205 45.30913 21.245588 69.37267 8.5071196 82.11114
206 44.43955 19.420256 69.45884 6.1758412 82.70326
207 43.56997 17.599217 69.54073 3.8511288 83.28882
208 42.70039 15.781515 69.61927 1.5315205 83.86927
209 41.83082 13.966326 69.69531 -0.7842448 84.44588
210 40.96124 12.152934 69.76954 -3.0972610 85.01974
211 40.09166 10.340716 69.84261 -5.4084829 85.59180
212 39.22208 8.529124 69.91504 -7.7187474 86.16291
213 38.35250 6.717675 69.98733 -10.0287917 86.73380
214 37.48293 4.905945 70.05991 -12.3392679 87.30512
215 36.61335 3.093553 70.13314 -14.6507544 87.87745
216 35.74377 1.280164 70.20738 -16.9637667 88.45131
217 34.87419 -0.534523 70.28291 -19.2787648 89.02715
218 34.00461 -2.350779 70.36001 -21.5961611 89.60539
219 33.13504 -4.168844 70.43892 -23.9163256 90.18640
220 32.26546 -5.988937 70.51985 -26.2395913 90.77051
221 31.39588 -7.811255 70.60302 -28.5662585 91.35802
222 30.52630 -9.635974 70.68858 -30.8965985 91.94920
223 29.65672 -11.463254 70.77670 -33.2308566 92.54431
224 28.78715 -13.293243 70.86754 -35.5692552 93.14355
225 27.91757 -15.126070 70.96121 -37.9119963 93.74713
226 27.04799 -16.961857 71.05784 -40.2592630 94.35524
227 26.17841 -18.800712 71.15754 -42.6112221 94.96805
228 25.30884 -20.642734 71.26040 -44.9680254 95.58570
229 24.43926 -22.488014 71.36653 -47.3298109 96.20832
230 23.56968 -24.336634 71.47599 -49.6967045 96.83606
231 22.70010 -26.188669 71.58887 -52.0688212 97.46902
232 21.83052 -28.044188 71.70523 -54.4462656 98.10731
233 20.96095 -29.903253 71.82514 -56.8291332 98.75102
Ljung-Box Tests
Box Test for modelfit_after_covid$residuals.
To check for correlations between successive forecast errors, we can make a correlogram and use the Ljung-Box test.
Box-Ljung test
data: modelfit_after_covid$residuals
X-squared = 1.6058e-05, df = 1, p-value = 0.9968
Box-Ljung test
data: modelfit_before_covid$residuals
X-squared = 0.0047812, df = 1, p-value = 0.9449
Here, the p value for both the models is greater than 0.05 . Hence, at a significance level of 0.05 we fail to reject the null hypothesis and conclude that the residual follows white noise. This means that the model fits the data well.
Artificial neural networks are very powerful and popular machine-learning algorithms that mimic how a brain works in order find patterns in your data.
In this project, we will build a basic neural network to model the price change of Amazon stocks in real Time.
ANNs make predictions by sending the inputs (in our case, the indicators) through the network of neurons, with the neurons firing off depending on the weights of the incoming signals. The final output is determined by the strength of the signals coming from the previous layer of neurons.
Data Import
Use “quantmod” package to download information for Amazon stocks.
Let’s see how we can quickly build a strategy using 4 technical indicators to see whether today’s price of Amazon’s stock .The 4 technical indicators are:
[1] "AMZN"
One of the most important procedures when forming a neural network is data normalization. This involves adjusting the data to a common scale so as to accurately compare predicted and actual values. Failure to normalize the data will typically result in the prediction value remaining the same across all observations, regardless of the input values.
Max-Min Normalization: For this method, we invoke the following function to normalize our data:
Data slicing:
'data.frame': 5209 obs. of 5 variables:
$ RSI3 : num 0.0853 0.0409 0.0402 0.6396 0.4188 ...
$ EMAcross : num 0.538 0.534 0.539 0.553 0.545 ...
$ MACDsignal: num 0.727 0.705 0.681 0.661 0.643 ...
$ BollingerB: num 0.423 0.353 0.362 0.477 0.408 ...
$ Price : num 0.417 0.424 0.444 0.422 0.433 ...
nnetGrid <- expand.grid(size = seq(from = 1, to = 5, by = 1)
,decay = seq(from = 0.1, to = 0.2, by = 0.1)
)
str(subTrain)
'data.frame': 12507 obs. of 5 variables:
$ RSI3 : num 0.0853 0.0409 0.0402 0.4188 0.3651 ...
$ EMAcross : num 0.538 0.534 0.539 0.545 0.544 ...
$ MACDsignal: num 0.727 0.705 0.681 0.643 0.625 ...
$ BollingerB: num 0.423 0.353 0.362 0.408 0.386 ...
$ Price : num 0.417 0.424 0.444 0.433 0.426 ...
nn_model <- train(Price~RSI3+EMAcross+MACDsignal+BollingerB, subTrain,
method = "nnet", algorithm = 'backprop',
trControl= TrainingParameters,
preProcess=c("scale","center"),
na.action = na.omit,
tuneGrid = nnetGrid,
trace=FALSE,
verbose=FALSE)
size decay RMSE Rsquared MAE RMSESD RsquaredSD
1 1 0.1 0.03933584 0.01238798 0.01586244 0.002504076 0.01443134
2 1 0.2 0.03934006 0.01239107 0.01583777 0.002507265 0.01451383
3 2 0.1 0.03933346 0.01268185 0.01586449 0.002503906 0.01468389
4 2 0.2 0.03933083 0.01281302 0.01584319 0.002506894 0.01488929
5 3 0.1 0.03933819 0.01287279 0.01586899 0.002504096 0.01467428
6 3 0.2 0.03933054 0.01293707 0.01584766 0.002504418 0.01501688
7 4 0.1 0.03932220 0.01314280 0.01584623 0.002496180 0.01459575
8 4 0.2 0.03932984 0.01300564 0.01584778 0.002504590 0.01508909
9 5 0.1 0.03932381 0.01361986 0.01584481 0.002522508 0.01473453
10 5 0.2 0.03932938 0.01304931 0.01584684 0.002504771 0.01512491
MAESD
1 0.0007305646
2 0.0007313067
3 0.0007312625
4 0.0007335536
5 0.0007326783
6 0.0007321606
7 0.0007377006
8 0.0007321347
9 0.0007322239
10 0.0007327730
actual prediction
7 0.4362379 0.4274795
10 0.4256909 0.4273008
97 0.4278731 0.4284938
139 0.4238725 0.4280213
483 0.4315391 0.4283605
783 0.4282804 0.4271653
896 0.4283095 0.4268096
908 0.4300552 0.4267116
911 0.4315391 0.4280867
952 0.4203374 0.4286907
1012 0.4257782 0.4279932
1711 0.4255164 0.4265486
1746 0.4299679 0.4280298
1804 0.4386092 0.4280147
2058 0.4317427 0.4280102
2073 0.4374163 0.4289039
2176 0.4056444 0.4292499
2276 0.4372417 0.4271038
2695 0.4251672 0.4280948
2746 0.4161477 0.4282272
2772 0.4228978 0.4272442
2784 0.4386092 0.4272447
3057 0.4236834 0.4273921
3139 0.4282513 0.4273274
3178 0.4408495 0.4281902
3220 0.4289496 0.4283887
3335 0.4316845 0.4266736
3443 0.4418097 0.4276801
3503 0.4245853 0.4275239
4018 0.4396857 0.4311302
4237 0.4353504 0.4281576
4261 0.4384055 0.4282462
4421 0.4145185 0.4284854
4526 0.5582194 0.4272961
4664 0.4444863 0.4258982
4786 0.4411985 0.4281155
4843 0.3776257 0.4279618
4865 0.4139076 0.4277111
4937 0.4610998 0.4286968
5030 0.5170493 0.4270784
99% accuracy is acheived to predict the price using the stock technical indicators.
RSI3 EMAcross MACDsignal BollingerB Price
7 0.32459911 0.5442847 0.6075494 0.3572573 0.4362379
10 0.20618845 0.5412815 0.5634201 0.3204323 0.4256909
97 0.24607269 0.5460137 0.2333221 0.3782227 0.4278731
139 0.84671114 0.5597144 0.7114260 0.7928638 0.4238725
483 0.80588690 0.5503068 0.6449523 0.7104961 0.4315391
783 0.54896975 0.5481855 0.7942074 0.5794676 0.4282804
896 0.36179313 0.5467044 0.7485756 0.4977961 0.4283095
908 0.36059256 0.5461656 0.7215884 0.5917109 0.4300552
911 0.87072321 0.5542228 0.7250983 0.8051809 0.4315391
952 0.72799440 0.5495230 0.5471357 0.5809809 0.4203374
1012 0.25508071 0.5462964 0.4156671 0.3042368 0.4257782
1711 0.17516899 0.5466259 0.7149182 0.3455139 0.4255164
1746 0.42878584 0.5473307 0.5385448 0.3833825 0.4299679
1804 0.95547536 0.5649359 0.7302750 0.9376553 0.4386092
2058 0.75475738 0.5528730 0.6755122 0.7233058 0.4317427
2073 0.85834756 0.5557843 0.6024193 0.5896256 0.4374163
2176 0.61647522 0.5560750 0.3465414 0.4212848 0.4056444
2276 0.38169960 0.5460403 0.7248211 0.4346285 0.4372417
2695 0.79630557 0.5568765 0.7103222 0.6807527 0.4251672
2746 0.86740868 0.5570476 0.6578077 0.8334958 0.4161477
2772 0.33325260 0.5425081 0.6137948 0.4871775 0.4228978
2784 0.11853439 0.5357345 0.5229431 0.2911652 0.4386092
3057 0.37948310 0.5450882 0.6474615 0.4269868 0.4236834
3139 0.47640198 0.5473919 0.6464690 0.6264217 0.4282513
3178 0.64602849 0.5532853 0.6476852 0.4856036 0.4408495
3220 0.78578515 0.5543646 0.6158716 0.6913156 0.4289496
3335 0.14403635 0.5345840 0.5990995 0.4992964 0.4316845
3443 0.95580068 0.6123559 0.6557729 0.9230118 0.4418097
3503 0.48426281 0.5469840 0.6377598 0.5614055 0.4245853
4018 0.04419073 0.4504553 0.4264717 0.1988550 0.4396857
4237 0.59029580 0.5558939 0.5694519 0.5221716 0.4353504
4261 0.87353837 0.5836426 0.6483177 0.7248558 0.4384055
4421 0.88473691 0.5823825 0.5743706 0.7673681 0.4145185
4526 0.59590136 0.5793888 0.7331421 0.5560403 0.5582194
4664 0.98636455 0.6589197 0.6674518 0.8129811 0.4444863
4786 0.62740722 0.5626872 0.5948031 0.5374183 0.4411985
4843 0.54126423 0.5618588 0.6234117 0.4116830 0.3776257
4865 0.63945676 0.5788562 0.5987941 0.6760626 0.4139076
4937 0.20789866 0.4876023 0.5509471 0.2674240 0.4610998
5030 0.44684424 0.5332037 0.7226938 0.6420079 0.5170493
[1] 0.0034655906 -0.0006396635 -0.0002464325 -0.0016497116 0.0012600367
[6] 0.0004426085 0.0005953472 0.0013262634 0.0013685815 -0.0033262027
[11] -0.0008800795 -0.0004101826 0.0007687922 0.0041881569 0.0014795282
[16] 0.0033666456 -0.0094548054 0.0040098370 -0.0011635185 -0.0048179652
[21] -0.0017289311 0.0044925247 -0.0014748155 0.0003667483 0.0049999668
[26] 0.0002225924 0.0019863283 0.0055785501 -0.0011681378 0.0033806843
[31] 0.0028470762 0.0040164223 -0.0055743799 0.0494191079 0.0073310991
[36] 0.0051666092 -0.0203900965 -0.0055105203 0.0126964783 0.0344970986
[1] 0.9974791
nnet variable importance
Overall
EMAcross 100.000
MACDsignal 11.556
BollingerB 6.033
RSI3 0.000
#### Graphical Representation of our Neural Network
From the above implementation, the results are impressive(99% accuracy) and convincing in terms of using a machine learning algorithm to decide on the price of the stock Majority of the attributes in the dataset contribute significantly to the building of a predictive model.
colnames(DataSetWalmart)
1 Class
2 RSI
3 EMAcross
4 MACD
5 SMI
6 WPR
7 ADX
8 CCI
9 CMO
10 ROC
Caret package provides train() method for training our data for various algorithms. We just need to pass different parameter values for different algorithms. Before train() method, we will first use trainControl() method.
We are setting 3 parameters of trainControl() method. The “method” parameter holds the details about resampling method. We can set “method” with many values like “boot”, “boot632”, “cv”, “repeatedcv”, “LOOCV”, “LGOCV” etc. For this project, let’s try to use repeatedcv i.e, repeated cross-validation.
The “number” parameter holds the number of resampling iterations. The “repeats” parameter contains the complete sets of folds to compute for our repeated cross-validation. We are using setting number =10 and repeats =3. This trainControl() methods returns a list. We are going to pass this on our train() method.
Before training our SVM classifier, set.seed().
For training SVM classifier, train() method should be passed with “method” parameter as “svmLinear”. We are passing our target variable Term_Deposit. The “Term_Deposit.~.” denotes a formula for using all attributes in our classifier and Term_Deposit. as the target variable. The “trControl” parameter should be passed with results from our trianControl() method. The “preProcess” parameter is for preprocessing our training data.
As discussed earlier for our data, preprocessing is a mandatory task. We are passing 2 values in our “preProcess” parameter “center” & “scale”. These two help for centering and scaling the data. After preProcessing these convert our training data with mean value as approximately “0” and standard deviation as “1”. The “tuneLength” parameter holds an integer value. This is for tuning our algorithm.
Support Vector Machines with Linear Kernel
18651 samples
7 predictor
2 classes: 'DOWN', 'UP'
Pre-processing: centered (7), scaled (7)
Resampling: Cross-Validated (10 fold, repeated 1 times)
Summary of sample sizes: 16786, 16786, 16786, 16786, 16786, 16786, ...
Resampling results across tuning parameters:
C Accuracy Kappa
0.25 0.8209209 0.6412469
0.50 0.8211354 0.6416789
Accuracy was used to select the optimal model using the largest value.
The final value used for the model was C = 0.5.
predictionsvm DOWN UP
DOWN 28 8
UP 5 28
[1] 0.8115942
Confusion Matrix and Statistics
Reference
Prediction DOWN UP
DOWN 28 8
UP 5 28
Accuracy : 0.8116
95% CI : (0.6994, 0.8957)
No Information Rate : 0.5217
P-Value [Acc > NIR] : 5.253e-07
Kappa : 0.6239
Mcnemar's Test P-Value : 0.5791
Sensitivity : 0.8485
Specificity : 0.7778
Pos Pred Value : 0.7778
Neg Pred Value : 0.8485
Prevalence : 0.4783
Detection Rate : 0.4058
Detection Prevalence : 0.5217
Balanced Accuracy : 0.8131
'Positive' Class : DOWN
Support Vector Machines with Radial Basis Function Kernel
18651 samples
7 predictor
2 classes: 'DOWN', 'UP'
Pre-processing: centered (7), scaled (7)
Resampling: Cross-Validated (10 fold, repeated 1 times)
Summary of sample sizes: 16786, 16786, 16786, 16786, 16786, 16786, ...
Resampling results across tuning parameters:
C Accuracy Kappa
0.25 0.8565757 0.7127826
0.50 0.8618302 0.7233515
Tuning parameter 'sigma' was held constant at a value of 0.5
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were sigma = 0.5 and C = 0.5.
[1] 0.7971014
Call:
summary.resamples(object = algo_results)
Models: SVM_RADIAL, SVM_LINEAR
Number of resamples: 10
Accuracy
Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
SVM_RADIAL 0.8509383 0.8571046 0.8630027 0.8618302 0.8657375 0.8723861 0
SVM_LINEAR 0.8144772 0.8171582 0.8209115 0.8211354 0.8248692 0.8284182 0
Kappa
Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
SVM_RADIAL 0.7011356 0.7139651 0.7255579 0.7233515 0.7312067 0.7446252 0
SVM_LINEAR 0.6280912 0.6334363 0.6410743 0.6416789 0.6492305 0.6567207 0
From the above implementation, the results are impressive and convincing in terms of using a machine learning algorithm to decide on the price change of walmart . Majority of the attributes in the dataset contribute significantly to the building of a predictive model. All the two SVM approach acheives good accuracy rate(>80%) and are easier to implement.
We will be trying to understand sentiment of tweets about the company Apple, By using the twitter for better understand public perception, Apple wants to monitor how people feel over time and how people receive new announcements.
Our challenge in this project is to see if we can correctly classify tweets as being negative, positive, or neither about Apple.
Sentiment Mining - Apple Stock
Preprocessing of Text data
Data
To collect the data needed for this task, we had to perform two steps.
The following graph shows the distribution of the number of tweets classified into each of the categories. We can see here that the majority of tweets were classified as neutral, with a small number classified as strongly negative or strongly positive.
Tweet
1 I have to say, Apple has by far the best customer care service I have ever received! @Apple @AppStore
2 iOS 7 is so fricking smooth & beautiful!! #ThanxApple @Apple
3 LOVE U @APPLE
4 Thank you @apple, loving my new iPhone 5S!!!!! #apple #iphone5S pic.twitter.com/XmHJCU4pcb
5 .@apple has the best customer service. In and out with a new phone in under 10min!
6 @apple ear pods are AMAZING! Best sound from in-ear headphones I've ever had!
Avg
1 2.0
2 2.0
3 1.8
4 1.8
5 1.8
6 1.8
One of fundamental concepts in text analysis, implemented in the package tm as well, is that of a corpus.A corpus is a collection of documents.
We will need to convert our tweets to a corpus for pre-processing. Various function in the tm package can be used to create a corpus in many different ways.
We will create it from the tweet column of our data frame using two functions, Corpus() and VectorSource(). We feed to this latter the Tweets variable of the tweets data frame.
Pre-processing steps
To deal with text data following pre-processing is required.Follow the standard steps to build and pre-process the corpus:
Build a document term matrix from the corpus, called dtm. Each operation, like stemming or removing stop words, can be done with one line in R, where we use the tm_map() function which takes as its first argument the name of a corpus and as second argument a function performing the transformation that we want to apply to the text.
'data.frame': 61 obs. of 1 variable:
$ text: chr " say appl far best custom care servic ever receiv appl appstor" "io frick smooth beauti thanxappl appl" "love u appl" "thank appl love new iphon s appl iphon pictwittercomxmhjcupcb" ...
<<DocumentTermMatrix (documents: 1181, terms: 3198)>>
Non-/sparse entries: 10102/3766736
Sparsity : 100%
Maximal term length: 99
Weighting : term frequency (tf)
Sample :
Terms
Docs appl get ipad iphon ipod itun make new phone whi
103 1 0 0 0 0 0 0 0 0 0
1066 2 0 1 0 0 1 0 0 0 0
215 3 0 2 0 0 0 0 0 0 0
506 3 0 0 0 0 0 0 0 0 0
592 1 0 0 0 0 0 0 0 1 1
62 1 1 0 0 0 0 0 0 1 1
740 1 1 0 0 0 0 0 1 0 0
756 1 0 0 0 0 0 0 0 0 0
86 1 0 0 2 0 0 0 1 0 0
900 1 0 0 0 0 0 0 0 0 0
Building machine learning model
Making predictions
model <- naive_bayes(as.data.frame(as.matrix(dtm_train)), dataset_train$positive, laplace = 1)
Confusion Matrix and Statistics
Reference
Prediction FALSE TRUE
FALSE 237 0
TRUE 65 0
Accuracy : 0.7848
95% CI : (0.7341, 0.8298)
No Information Rate : 1
P-Value [Acc > NIR] : 1
Kappa : 0
Mcnemar's Test P-Value : 2.051e-15
Sensitivity : 0.7848
Specificity : NA
Pos Pred Value : NA
Neg Pred Value : NA
Prevalence : 1.0000
Detection Rate : 0.7848
Detection Prevalence : 0.7848
Balanced Accuracy : NA
'Positive' Class : FALSE
Cell Contents
|-------------------------|
| N |
|-------------------------|
Total Observations in Table: 302
| dataset_test$positive
model_predict | FALSE | Row Total |
--------------|-----------|-----------|
FALSE | 237 | 237 |
--------------|-----------|-----------|
TRUE | 65 | 65 |
--------------|-----------|-----------|
Column Total | 302 | 302 |
--------------|-----------|-----------|
[1] "appl" "iphon" "new" "phone" "whi" "get" "ipad" "itun" "ipod"
Stock market prediction is an incredibly difficult task, due to the randomness and noisiness found in the market. Yet, predicting market behaviors is a very important task. Correctly predicting stock price directions can be used to maximize profits, as well as to minimize risk. There are two types of methods to predicting market behavior. One is predicting the future price of an asset. This is usually done using time series analysis to fit a specific model, like ARIMA or GARCH, to some historical data. The other is predicting the future trend of an asset. That is, whether one thinks it will go up or down in price, treating it as a classification problem.
The goal of this project is to create an intelligent model, using the Random Forest model, that can correctly forecast the behavior of a stock’s price n days out.
Data Import
Use “quantmod” package to download information for Amazon stocks.
The data used for this project consists of regular stock data (open, close, volume, etc.) from Yahoo finance, and ranges from the year 2000 to 2018.
From this data, technical indicators were calculated for every stock. Below are all the technical indicators used for this model:
- Relative Strength Index
- Stochastic Oscillator
- William %R
- Moving Average Convergence Divergence
- Price Rate of Change
- On Balance VolumeThe last step of pre-processing the data was calculating the response variable. Since we are treating this as a classification problem, the response variable was binary. The equation for calculating the response variable is below:Response=Closet+n−Closet
It states that the adjusted close price at t+n, where n is the number of days out you want to predict, minus the current adjusted close price will map to a value that says the stock price went up from the point at time t, or that it went down.
[1] "^GSPC"
colnames(dataset1)
1 rsi
2 EMA
3 signal
4 pctB
5 GSPC.Close
Check for missing data in relativeStrengthIndex20, exponentialMovingAverage20, MACDsignal, PercentageChngpctB, Price. Omit the n/a values in dataset.
Print the number of missed value for each attribute in dataset.
[1] "rsi"
[1] 20
[1] "EMA"
[1] 19
[1] "signal"
[1] 33
[1] "pctB"
[1] 19
[1] "GSPC.Close"
[1] 0
RandoM Forest Machine Learning Model
Split the dataset into training & test dataset.80 % of the data is training data.20 % of the data is test dataset.
Feature Scaling -> Normalization /Scale and dropping the feature varaibles.
# Feature Scaling (Normalization and dropping the predicted variable)
training_set[-5] = scale(training_set[-5])
test_set[-5] = scale(test_set[-5])
Length Class Mode
call 4 -none- call
type 1 -none- character
predicted 1884 factor numeric
err.rate 30 -none- numeric
confusion 6 -none- numeric
votes 3768 matrix numeric
oob.times 1884 -none- numeric
classes 2 -none- character
importance 4 -none- numeric
importanceSD 0 -none- NULL
localImportance 0 -none- NULL
proximity 0 -none- NULL
ntree 1 -none- numeric
mtry 1 -none- numeric
forest 14 -none- list
y 1884 factor numeric
test 0 -none- NULL
inbag 0 -none- NULL
predict_val
0 1
0 89 128
1 102 152
[1] "Model Accuracy is"
[1] 0.5116773
Random Forest
1884 samples
4 predictor
2 classes: '0', '1'
No pre-processing
Resampling: Cross-Validated (3 fold)
Summary of sample sizes: 1257, 1256, 1255
Resampling results across tuning parameters:
mtry Accuracy Kappa
3 0.5058141 -0.003131317
6 0.5021045 -0.009128875
9 0.5132502 0.013836823
Kappa was used to select the optimal model using the largest value.
The final value used for the model was mtry = 9.
This is a beginning of the use of ML algorithms for predicting the time series data or the stock prices.It can be modified and optimized in a lot of ways to produce much better and much more efficient and accurate results.
Choosing the right technical indicators which will influence the price change is daunting.In this project we tried to predict the price change through variety of technical indicators in different ML algorithms.
Although we have acheived accuracy of 99 percent in few ML algorithms, there are many other things that impact the prices of stocks such as:Political and social upheavals ,current affairs etc
Thus, we can say stock market price change is quite a dynamic movement.